# DOMAIN: Botanical Research
• CONTEXT: University X is currently undergoing some research involving understanding the characteristics of plant and plant seedlings at
various stages of growth. They already have have invested on curating sample images. They require an automation which can create a
classifier capable of determining a plant's species from a photo.
• DATA DESCRIPTION: The dataset comprises of images from 12 plant species.
Source: https://www.kaggle.com/c/plant-seedlings-classification/data.
• PROJECT OBJECTIVE: To create a classifier capable of determining a plant's species from a photo.
In [1]:
!pip install keras-tuner
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting keras-tuner
  Downloading keras_tuner-1.1.3-py3-none-any.whl (135 kB)
     |████████████████████████████████| 135 kB 7.5 MB/s 
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from keras-tuner) (1.21.6)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from keras-tuner) (2.8.0)
Requirement already satisfied: ipython in /usr/local/lib/python3.7/dist-packages (from keras-tuner) (7.9.0)
Collecting kt-legacy
  Downloading kt_legacy-1.0.4-py3-none-any.whl (9.6 kB)
Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from keras-tuner) (21.3)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from keras-tuner) (2.23.0)
Requirement already satisfied: pexpect in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (4.8.0)
Collecting jedi>=0.10
  Downloading jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)
     |████████████████████████████████| 1.6 MB 63.6 MB/s 
Requirement already satisfied: pickleshare in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (0.7.5)
Requirement already satisfied: decorator in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (4.4.2)
Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (5.1.1)
Requirement already satisfied: pygments in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (2.6.1)
Requirement already satisfied: backcall in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (0.2.0)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (2.0.10)
Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/dist-packages (from ipython->keras-tuner) (57.4.0)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from jedi>=0.10->ipython->keras-tuner) (0.8.3)
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython->keras-tuner) (1.15.0)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython->keras-tuner) (0.2.5)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->keras-tuner) (3.0.9)
Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.7/dist-packages (from pexpect->ipython->keras-tuner) (0.7.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->keras-tuner) (2022.6.15)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->keras-tuner) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->keras-tuner) (1.24.3)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->keras-tuner) (2.10)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (1.47.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (0.6.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (3.4.1)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (0.37.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (1.8.1)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (3.17.3)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (1.2.0)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard->keras-tuner) (1.0.1)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->keras-tuner) (4.9)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->keras-tuner) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->keras-tuner) (4.2.4)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->keras-tuner) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard->keras-tuner) (4.12.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard->keras-tuner) (4.1.1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard->keras-tuner) (3.8.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->keras-tuner) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->keras-tuner) (3.2.0)
Installing collected packages: jedi, kt-legacy, keras-tuner
Successfully installed jedi-0.18.1 keras-tuner-1.1.3 kt-legacy-1.0.4
In [2]:
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [3]:
import os
WORK_DIR ='/content/drive/My Drive/Colab/cv/week1-2/Assignment';
os.chdir(WORK_DIR)
In [4]:
from zipfile import ZipFile
import cv2
import pandas as pd
from random  import randint
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder
import numpy as np
from sklearn.model_selection import train_test_split

from tensorflow.keras.models import Sequential,Model, load_model
from tensorflow.keras.layers import Conv2D,MaxPooling2D,Dense,Flatten,Dropout,Activation, BatchNormalization, GlobalAveragePooling2D
from tensorflow.keras.optimizers import Adam, Adagrad
from tensorflow.keras.callbacks import EarlyStopping,ReduceLROnPlateau,ModelCheckpoint
from tensorflow.keras import regularizers
from sklearn.model_selection import GridSearchCV
from tensorflow.keras.losses import CategoricalCrossentropy,SparseCategoricalCrossentropy
import tensorflow as tf
from tensorflow import keras
from kerastuner import Hyperband
import kerastuner as kerastuner
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.utils import to_categorical, array_to_img
from PIL import Image
from PIL import ImageFilter
import random
from sklearn.decomposition import PCA
import tensorflow as tf
seed = 42
random.seed(seed)
tf.random.set_seed(seed)
from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics
from sklearn.metrics import confusion_matrix, classification_report
import seaborn as sns

np.random.seed(seed)
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `import kerastuner` is deprecated, please use `import keras_tuner`.
  1. Import and Understand the data [12 Marks]

A. Extract ‘plant-seedlings-classification.zip’ into new folder (unzipped) using python. [2 Marks]


In [5]:
PLANT_SEEDLING_PATH = './plant-seedlings-classification.zip'
with ZipFile(PLANT_SEEDLING_PATH, 'r') as zipfile:
  zipfile.extractall()

B. Map the images from train folder with train labels to form a DataFrame. [6 Marks]

In [6]:
root = WORK_DIR + '/plant-seedlings-classification/train'
cols = ['Image Name', 'class', 'Actual Image']
N = 4
plant_seed_df = pd.DataFrame(columns = cols)
for path, subdirs, files in os.walk(root):
    for name in files:
        if not name.startswith('.'):
          full_path = os.path.join(path, name);
          #print('Reading path : '+ full_path)
          img = cv2.imread(full_path)
          img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
          plant_seed_df = plant_seed_df.append({'Image Name': name, 'class':os.path.basename(os.path.dirname(full_path)) ,'Actual Image': img},ignore_index=True)
In [7]:
plant_seed_df.head(5)
Out[7]:
Image Name class Actual Image
0 6a4ef17c2.png Cleavers [[[72, 61, 45], [73, 66, 50], [49, 47, 28], [4...
1 0515bc601.png Cleavers [[[161, 156, 146], [162, 156, 146], [162, 156,...
2 0ac327873.png Cleavers [[[91, 78, 67], [88, 73, 63], [88, 74, 63], [9...
3 94f82e916.png Cleavers [[[80, 62, 45], [78, 57, 42], [81, 60, 46], [8...
4 2c5c4d127.png Cleavers [[[137, 98, 62], [135, 96, 59], [136, 97, 62],...

C. Write a function that will select n random images and display images along with its species. [4 Marks]


In [8]:
# generate index of each class 
def generate_random_index(totalNo, total_no_of_records, label):
  random_index = [];
  uniqueLabel = np.unique(label)
  for index , category in  enumerate(uniqueLabel):
    if index >=totalNo:
      break; 
    random_index.append(random.choice(plant_seed_df[plant_seed_df['class'] == category].index))
  while(len(random_index) != totalNo):
    random_index.append(random.randint(0, total_no_of_records));

  return random_index;
In [9]:
# Display Image with label
def displayImage(input,y=None):
  if (isinstance(input, pd.DataFrame)):
    for i in range(len(input)):
      plt.imshow(input.iloc[i, 2])
      plt.xlabel(input.iloc[i, 1])
      plt.show()
  else:
    for i in range(len(input)):
      plt.imshow(input[i])
      plt.xlabel("Actual Label: " + str(y[i]))
      plt.show();
In [10]:
#Select 5 images
TOTAL_NUMBER_OF_RANDOM_IMAGE = 5
# Take ranom 5 index
random_plant_seed_index= generate_random_index(TOTAL_NUMBER_OF_RANDOM_IMAGE, len(plant_seed_df), plant_seed_df['class'])
#Pick up the records based on the dataset
random_plant_seed_df = plant_seed_df.iloc[random_plant_seed_index]
print('rows selected ')
# display it
display(random_plant_seed_df)
# Print the image
displayImage(random_plant_seed_df)
rows selected 
Image Name class Actual Image
3437 9e2bfa93d.png Black-grass [[[65, 52, 50], [68, 56, 54], [68, 58, 55], [6...
4151 6aae02cb3.png Charlock [[[85, 68, 48], [84, 65, 47], [81, 61, 44], [8...
140 f5608b615.png Cleavers [[[86, 64, 41], [88, 66, 43], [85, 63, 39], [8...
922 a6d54c45c.png Common Chickweed [[[95, 78, 64], [92, 75, 61], [87, 72, 56], [7...
3216 9efa077b7.png Common wheat [[[82, 53, 33], [74, 51, 29], [75, 56, 35], [7...

Observations: Images are of un-equal length.


2. Data preprocessing

A. Create X & Y from the DataFrame


In [11]:
X = plant_seed_df[['Actual Image']]
y = plant_seed_df[['class']]

B. Encode labels of the images


In [12]:
label_encoder = LabelEncoder()
y_le = label_encoder.fit_transform(y.values.ravel())
y_le
y_le = to_categorical(y_le, num_classes = 12)
In [13]:
mapping = dict(zip(label_encoder.classes_,label_encoder.transform(label_encoder.classes_)))
mapping
Out[13]:
{'Black-grass': 0,
 'Charlock': 1,
 'Cleavers': 2,
 'Common Chickweed': 3,
 'Common wheat': 4,
 'Fat Hen': 5,
 'Loose Silky-bent': 6,
 'Maize': 7,
 'Scentless Mayweed': 8,
 'Shepherds Purse': 9,
 'Small-flowered Cranesbill': 10,
 'Sugar beet': 11}

Use dictionary to find out the name and its corresponding label assigned

C. Unify shape of all the images


In [14]:
# Resize the image into 256 x 256
def resize_images(img):

  img = np.array(img).astype(np.uint8)
  #print(img.dtype)
  res = cv2.resize(img,(256,256), interpolation = cv2.INTER_CUBIC)
  return res

#save resized images into images.
images = [resize_images(img) for img in X['Actual Image']]

Using randint function select image and check the shape . It should be 256 X 256

In [15]:
rndm_index = randint(1,len(plant_seed_df));
print(rndm_index)
images[rndm_index].shape
1144
Out[15]:
(256, 256, 3)

D. Normalise all the images. [2 Marks]

In [16]:
# Normalise the image and scale it with 0 min value and 1 max value
def normalize(img):
  img = np.expand_dims(img, axis = 0)
  img =img*1/255.0
  return img;

images = [resize_images(img) for img in images]

Checking the min and max of first image. It should be in 0 to 255 range

In [17]:
print('Min pixel  :  ' + str(np.min(images[0])) + '   and  Max pixel  : ' + str(np.max(images[0])))
Min pixel  :  0   and  Max pixel  : 186
  1. Model training
  A. Split the data into train and test data.
In [18]:
X_train, X_test, y_train, y_test = train_test_split(images, y_le, test_size=0.20, stratify=y, random_state=1)
In [19]:
# Print the shape of first image 
X_train[0].shape
Out[19]:
(256, 256, 3)
In [20]:
y_train.shape
Out[20]:
(3800, 12)
In [21]:
#see number of images in each label
X_train = np.array(X_train)
y_train = np.array(y_train)
X_test = np.array(X_test)
y_test = np.array(y_test)
print("images shape: ", X_train.shape)
print("classes shape: ", y_train.shape)
images shape:  (3800, 256, 256, 3)
classes shape:  (3800, 12)
In [22]:
# Reducing overfitting by Using data augmentaion, 
train_datagen = ImageDataGenerator(rotation_range=180,  
        zoom_range = 0.1,
        width_shift_range=0.5,  
        height_shift_range=0.5, 
        shear_range=0.2, 
        horizontal_flip=True,  
        vertical_flip=True,fill_mode='nearest')

train_datagen.fit(X_train)
In [23]:
tf.keras.backend.clear_session()

B. Create new CNN architecture to train the model

In [24]:
tf.keras.backend.clear_session()
In [25]:
stop_early = EarlyStopping(monitor='val_loss', mode='min', patience=7)
reduce_lr = ReduceLROnPlateau(monitor='val_loss',factor=0.1,patience=5,min_lr=0.00001,model='auto')
In [26]:
def build_model(hp):
    model=Sequential();
    model.add(tf.keras.Input(shape=(256,256,3)))
    for layer in range(hp.Int('num_conv_layer', 1,6)):
      model.add(BatchNormalization())
      model.add(Conv2D(filters = hp.Int('conv '+str(layer+1)+'_filter', min_value=32, max_value=128, step=16),kernel_size=hp.Choice('Conv_Kernel_'+str(layer+1), [3,5,7]), padding=hp.Choice('Conv_padding_'+str(layer+1), ['valid','same']) ,
                       activation= "relu" ))
      model.add(BatchNormalization())
      model.add(MaxPooling2D())
  
    model.add(GlobalAveragePooling2D())
    for layer in range(hp.Int('num__dense_layer', 1,4)):
      model.add(BatchNormalization())
      model.add(Dropout(rate=hp.Float('dropout_' + str(layer + 1) ,
                                      min_value=0.0,
                                      max_value=0.5,
                                      step=0.1)))
      model.add(Dense(units=hp.Int("Units_"+str(layer+1), min_value=32, max_value=512, step=64), activation= "relu",  kernel_initializer='he_uniform'))
      model.add(BatchNormalization())
      
    model.add(Dense(units=12,activation='softmax'))
    model.compile(optimizer='adam',
              loss=CategoricalCrossentropy(),
              metrics=['accuracy'])
    return model
In [27]:
tuner  = Hyperband(build_model, objective=kerastuner.Objective("val_accuracy", direction="max"), max_epochs=10, factor=2, hyperparameters=None, directory = 'CNN_project',project_name = 'Q1',overwrite=True)
tuner.search_space_summary()
Search space summary
Default search space size: 7
num_conv_layer (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 6, 'step': 1, 'sampling': None}
conv 1_filter (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 128, 'step': 16, 'sampling': None}
Conv_Kernel_1 (Choice)
{'default': 3, 'conditions': [], 'values': [3, 5, 7], 'ordered': True}
Conv_padding_1 (Choice)
{'default': 'valid', 'conditions': [], 'values': ['valid', 'same'], 'ordered': False}
num__dense_layer (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 4, 'step': 1, 'sampling': None}
dropout_1 (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None}
Units_1 (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 64, 'sampling': None}
In [28]:
# tuner.search(train_datagen.flow(X_train,y_train, batch_size=60),validation_data=(X_test,y_test), callbacks=[reduce_lr, stop_early], verbose=2,steps_per_epoch=(X_train.shape[0]/60),use_multiprocessing=True,workers=6)
  • The code has been commented. It is running code, but it is taking 60 min to hypertuned. In order to save time, i save the best hypertuned model into the disk and load it
  • The below code is saving the model into the hard disk
In [29]:
#tuner.results_summary()
In [30]:
# best_hps=tuner.get_best_hyperparameters(num_trials=1)[0]
# model = tuner.hypermodel.build(best_hps)
# model.save('Hypertuned_CNN_Prob11.h5')
# Above code is used to save the best model into the memory. Since it is taking too much time, so model has been saved and reuse further
In [31]:
model = load_model('Hypertuned_CNN_Prob11.h5')
In [32]:
model.summary()
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 batch_normalization_14 (Bat  (None, 256, 256, 3)      12        
 chNormalization)                                                
                                                                 
 conv2d_4 (Conv2D)           (None, 252, 252, 32)      2432      
                                                                 
 batch_normalization_15 (Bat  (None, 252, 252, 32)     128       
 chNormalization)                                                
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 126, 126, 32)     0         
 2D)                                                             
                                                                 
 batch_normalization_16 (Bat  (None, 126, 126, 32)     128       
 chNormalization)                                                
                                                                 
 conv2d_5 (Conv2D)           (None, 124, 124, 96)      27744     
                                                                 
 batch_normalization_17 (Bat  (None, 124, 124, 96)     384       
 chNormalization)                                                
                                                                 
 max_pooling2d_5 (MaxPooling  (None, 62, 62, 96)       0         
 2D)                                                             
                                                                 
 batch_normalization_18 (Bat  (None, 62, 62, 96)       384       
 chNormalization)                                                
                                                                 
 conv2d_6 (Conv2D)           (None, 60, 60, 96)        83040     
                                                                 
 batch_normalization_19 (Bat  (None, 60, 60, 96)       384       
 chNormalization)                                                
                                                                 
 max_pooling2d_6 (MaxPooling  (None, 30, 30, 96)       0         
 2D)                                                             
                                                                 
 batch_normalization_20 (Bat  (None, 30, 30, 96)       384       
 chNormalization)                                                
                                                                 
 conv2d_7 (Conv2D)           (None, 30, 30, 112)       526960    
                                                                 
 batch_normalization_21 (Bat  (None, 30, 30, 112)      448       
 chNormalization)                                                
                                                                 
 max_pooling2d_7 (MaxPooling  (None, 15, 15, 112)      0         
 2D)                                                             
                                                                 
 global_average_pooling2d_1   (None, 112)              0         
 (GlobalAveragePooling2D)                                        
                                                                 
 batch_normalization_22 (Bat  (None, 112)              448       
 chNormalization)                                                
                                                                 
 dropout_3 (Dropout)         (None, 112)               0         
                                                                 
 dense_4 (Dense)             (None, 480)               54240     
                                                                 
 batch_normalization_23 (Bat  (None, 480)              1920      
 chNormalization)                                                
                                                                 
 dense_5 (Dense)             (None, 12)                5772      
                                                                 
=================================================================
Total params: 704,808
Trainable params: 702,498
Non-trainable params: 2,310
_________________________________________________________________
# Training the hypertuned model with 30 epochs with data agumentatin dataset
In [33]:
model_checkpoint = ModelCheckpoint('Hyptun_plantspecies_CNN_model.h5', save_best_only=True, monitor='val_accuracy', mode='max', verbose=1)
history = model.fit(train_datagen.flow(X_train,y_train, batch_size=60),validation_data=(X_test,y_test), callbacks=[reduce_lr, stop_early, model_checkpoint], epochs=30)
Epoch 1/30
64/64 [==============================] - ETA: 0s - loss: 1.8856 - accuracy: 0.3932
Epoch 1: val_accuracy improved from -inf to 0.14526, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 64s 795ms/step - loss: 1.8856 - accuracy: 0.3932 - val_loss: 2.7144 - val_accuracy: 0.1453 - lr: 0.0010
Epoch 2/30
64/64 [==============================] - ETA: 0s - loss: 1.4177 - accuracy: 0.5268
Epoch 2: val_accuracy improved from 0.14526 to 0.22105, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 774ms/step - loss: 1.4177 - accuracy: 0.5268 - val_loss: 2.7655 - val_accuracy: 0.2211 - lr: 0.0010
Epoch 3/30
64/64 [==============================] - ETA: 0s - loss: 1.1580 - accuracy: 0.6071
Epoch 3: val_accuracy improved from 0.22105 to 0.34000, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 776ms/step - loss: 1.1580 - accuracy: 0.6071 - val_loss: 2.3353 - val_accuracy: 0.3400 - lr: 0.0010
Epoch 4/30
64/64 [==============================] - ETA: 0s - loss: 1.0446 - accuracy: 0.6458
Epoch 4: val_accuracy did not improve from 0.34000
64/64 [==============================] - 50s 779ms/step - loss: 1.0446 - accuracy: 0.6458 - val_loss: 1.9852 - val_accuracy: 0.3316 - lr: 0.0010
Epoch 5/30
64/64 [==============================] - ETA: 0s - loss: 0.9609 - accuracy: 0.6803
Epoch 5: val_accuracy improved from 0.34000 to 0.50000, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 779ms/step - loss: 0.9609 - accuracy: 0.6803 - val_loss: 1.5183 - val_accuracy: 0.5000 - lr: 0.0010
Epoch 6/30
64/64 [==============================] - ETA: 0s - loss: 0.8648 - accuracy: 0.7068
Epoch 6: val_accuracy improved from 0.50000 to 0.66211, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 779ms/step - loss: 0.8648 - accuracy: 0.7068 - val_loss: 1.0037 - val_accuracy: 0.6621 - lr: 0.0010
Epoch 7/30
64/64 [==============================] - ETA: 0s - loss: 0.8103 - accuracy: 0.7258
Epoch 7: val_accuracy did not improve from 0.66211
64/64 [==============================] - 49s 768ms/step - loss: 0.8103 - accuracy: 0.7258 - val_loss: 1.5097 - val_accuracy: 0.5495 - lr: 0.0010
Epoch 8/30
64/64 [==============================] - ETA: 0s - loss: 0.7527 - accuracy: 0.7358
Epoch 8: val_accuracy improved from 0.66211 to 0.72211, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 778ms/step - loss: 0.7527 - accuracy: 0.7358 - val_loss: 0.7995 - val_accuracy: 0.7221 - lr: 0.0010
Epoch 9/30
64/64 [==============================] - ETA: 0s - loss: 0.7083 - accuracy: 0.7582
Epoch 9: val_accuracy improved from 0.72211 to 0.74632, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 775ms/step - loss: 0.7083 - accuracy: 0.7582 - val_loss: 0.8136 - val_accuracy: 0.7463 - lr: 0.0010
Epoch 10/30
64/64 [==============================] - ETA: 0s - loss: 0.7054 - accuracy: 0.7626
Epoch 10: val_accuracy did not improve from 0.74632
64/64 [==============================] - 49s 769ms/step - loss: 0.7054 - accuracy: 0.7626 - val_loss: 0.7564 - val_accuracy: 0.7463 - lr: 0.0010
Epoch 11/30
64/64 [==============================] - ETA: 0s - loss: 0.6605 - accuracy: 0.7716
Epoch 11: val_accuracy did not improve from 0.74632
64/64 [==============================] - 49s 765ms/step - loss: 0.6605 - accuracy: 0.7716 - val_loss: 0.9265 - val_accuracy: 0.6916 - lr: 0.0010
Epoch 12/30
64/64 [==============================] - ETA: 0s - loss: 0.6010 - accuracy: 0.7958
Epoch 12: val_accuracy did not improve from 0.74632
64/64 [==============================] - 49s 769ms/step - loss: 0.6010 - accuracy: 0.7958 - val_loss: 0.8397 - val_accuracy: 0.7189 - lr: 0.0010
Epoch 13/30
64/64 [==============================] - ETA: 0s - loss: 0.5984 - accuracy: 0.8016
Epoch 13: val_accuracy improved from 0.74632 to 0.77684, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 777ms/step - loss: 0.5984 - accuracy: 0.8016 - val_loss: 0.6399 - val_accuracy: 0.7768 - lr: 0.0010
Epoch 14/30
64/64 [==============================] - ETA: 0s - loss: 0.5394 - accuracy: 0.8218
Epoch 14: val_accuracy improved from 0.77684 to 0.79053, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 775ms/step - loss: 0.5394 - accuracy: 0.8218 - val_loss: 0.5701 - val_accuracy: 0.7905 - lr: 0.0010
Epoch 15/30
64/64 [==============================] - ETA: 0s - loss: 0.5148 - accuracy: 0.8234
Epoch 15: val_accuracy improved from 0.79053 to 0.86526, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 774ms/step - loss: 0.5148 - accuracy: 0.8234 - val_loss: 0.3937 - val_accuracy: 0.8653 - lr: 0.0010
Epoch 16/30
64/64 [==============================] - ETA: 0s - loss: 0.5121 - accuracy: 0.8271
Epoch 16: val_accuracy did not improve from 0.86526
64/64 [==============================] - 49s 763ms/step - loss: 0.5121 - accuracy: 0.8271 - val_loss: 0.5375 - val_accuracy: 0.8105 - lr: 0.0010
Epoch 17/30
64/64 [==============================] - ETA: 0s - loss: 0.4694 - accuracy: 0.8387
Epoch 17: val_accuracy did not improve from 0.86526
64/64 [==============================] - 49s 768ms/step - loss: 0.4694 - accuracy: 0.8387 - val_loss: 0.4639 - val_accuracy: 0.8337 - lr: 0.0010
Epoch 18/30
64/64 [==============================] - ETA: 0s - loss: 0.4437 - accuracy: 0.8508
Epoch 18: val_accuracy did not improve from 0.86526
64/64 [==============================] - 49s 776ms/step - loss: 0.4437 - accuracy: 0.8508 - val_loss: 0.4239 - val_accuracy: 0.8579 - lr: 0.0010
Epoch 19/30
64/64 [==============================] - ETA: 0s - loss: 0.4432 - accuracy: 0.8495
Epoch 19: val_accuracy did not improve from 0.86526
64/64 [==============================] - 49s 769ms/step - loss: 0.4432 - accuracy: 0.8495 - val_loss: 0.4528 - val_accuracy: 0.8411 - lr: 0.0010
Epoch 20/30
64/64 [==============================] - ETA: 0s - loss: 0.4230 - accuracy: 0.8542
Epoch 20: val_accuracy did not improve from 0.86526
64/64 [==============================] - 50s 783ms/step - loss: 0.4230 - accuracy: 0.8542 - val_loss: 1.8872 - val_accuracy: 0.6042 - lr: 0.0010
Epoch 21/30
64/64 [==============================] - ETA: 0s - loss: 0.3639 - accuracy: 0.8755
Epoch 21: val_accuracy improved from 0.86526 to 0.90947, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 773ms/step - loss: 0.3639 - accuracy: 0.8755 - val_loss: 0.2604 - val_accuracy: 0.9095 - lr: 1.0000e-04
Epoch 22/30
64/64 [==============================] - ETA: 0s - loss: 0.3443 - accuracy: 0.8847
Epoch 22: val_accuracy improved from 0.90947 to 0.92632, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 776ms/step - loss: 0.3443 - accuracy: 0.8847 - val_loss: 0.2206 - val_accuracy: 0.9263 - lr: 1.0000e-04
Epoch 23/30
64/64 [==============================] - ETA: 0s - loss: 0.3156 - accuracy: 0.8921
Epoch 23: val_accuracy did not improve from 0.92632
64/64 [==============================] - 49s 766ms/step - loss: 0.3156 - accuracy: 0.8921 - val_loss: 0.2217 - val_accuracy: 0.9211 - lr: 1.0000e-04
Epoch 24/30
64/64 [==============================] - ETA: 0s - loss: 0.3097 - accuracy: 0.8942
Epoch 24: val_accuracy improved from 0.92632 to 0.93474, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 774ms/step - loss: 0.3097 - accuracy: 0.8942 - val_loss: 0.2015 - val_accuracy: 0.9347 - lr: 1.0000e-04
Epoch 25/30
64/64 [==============================] - ETA: 0s - loss: 0.3238 - accuracy: 0.8895
Epoch 25: val_accuracy did not improve from 0.93474
64/64 [==============================] - 49s 764ms/step - loss: 0.3238 - accuracy: 0.8895 - val_loss: 0.2463 - val_accuracy: 0.9095 - lr: 1.0000e-04
Epoch 26/30
64/64 [==============================] - ETA: 0s - loss: 0.3043 - accuracy: 0.8979
Epoch 26: val_accuracy did not improve from 0.93474
64/64 [==============================] - 49s 769ms/step - loss: 0.3043 - accuracy: 0.8979 - val_loss: 0.2172 - val_accuracy: 0.9263 - lr: 1.0000e-04
Epoch 27/30
64/64 [==============================] - ETA: 0s - loss: 0.2834 - accuracy: 0.9097
Epoch 27: val_accuracy improved from 0.93474 to 0.94000, saving model to Hyptun_plantspecies_CNN_model.h5
64/64 [==============================] - 50s 774ms/step - loss: 0.2834 - accuracy: 0.9097 - val_loss: 0.2142 - val_accuracy: 0.9400 - lr: 1.0000e-04
Epoch 28/30
64/64 [==============================] - ETA: 0s - loss: 0.3009 - accuracy: 0.9034
Epoch 28: val_accuracy did not improve from 0.94000
64/64 [==============================] - 49s 768ms/step - loss: 0.3009 - accuracy: 0.9034 - val_loss: 0.2001 - val_accuracy: 0.9389 - lr: 1.0000e-04
Epoch 29/30
64/64 [==============================] - ETA: 0s - loss: 0.2769 - accuracy: 0.9042
Epoch 29: val_accuracy did not improve from 0.94000
64/64 [==============================] - 49s 764ms/step - loss: 0.2769 - accuracy: 0.9042 - val_loss: 0.2020 - val_accuracy: 0.9337 - lr: 1.0000e-04
Epoch 30/30
64/64 [==============================] - ETA: 0s - loss: 0.2733 - accuracy: 0.9111
Epoch 30: val_accuracy did not improve from 0.94000
64/64 [==============================] - 49s 763ms/step - loss: 0.2733 - accuracy: 0.9111 - val_loss: 0.2034 - val_accuracy: 0.9358 - lr: 1.0000e-04
  • Plot the Accuracy and Loss of train and test
In [34]:
# Plot the loss and accuracy curves for training and validation 
fig, ax = plt.subplots(2,1)
ax[0].plot(history.history['loss'], color='b', label="Training loss")
ax[0].plot(history.history['val_loss'], color='r', label="validation loss",axes =ax[0])
legend = ax[0].legend(loc='best', shadow=True)

ax[1].plot(history.history['accuracy'], color='b', label="Training accuracy")
ax[1].plot(history.history['val_accuracy'], color='r',label="Validation accuracy")
legend = ax[1].legend(loc='best', shadow=True)
In [35]:
eval_result = model.evaluate(X_test, y_test)
print("[test loss, test accuracy]:", eval_result)
30/30 [==============================] - 1s 19ms/step - loss: 0.2034 - accuracy: 0.9358
[test loss, test accuracy]: [0.2033836543560028, 0.9357894659042358]

Model is giving 95% Accuracy

D. Select a random image and print actual label and predicted label for the same.

In [36]:
random_number = randint(0,len(X_test))
img = X_test[random_number]
actual_label = y_test[random_number]
plt.imshow(img)
img = np.expand_dims(img, axis = 0)
#img =img*1/255.0
#Check the size of the Image array again
print('After expand_dims: '+ str(img.shape))
result = model.predict(img)
predicted_index = np.argmax(result[0]);
key_list = list(mapping.keys())
val_list = list(mapping.values())
# print key with val 100
position = val_list.index(predicted_index)
predicted = key_list[position]

actual = key_list[np.argmax(actual_label, axis=None, out=None)]
plt.suptitle("Actual label "+ actual +" ,,,,, Predicted label "+predicted)
plt.show()
After expand_dims: (1, 256, 256, 3)

Removing the variables to free RAM


In [37]:
tf.keras.backend.clear_session()
del plant_seed_df, random_plant_seed_index,random_plant_seed_df,X,y, y_le,mapping, rndm_index, images, X_train, X_test, y_train, y_test ,train_datagen, tuner,model, random_number, predicted, actual, key_list,val_list

DOMAIN: Botanical Research • CONTEXT: University X is currently undergoing some research involving understanding the characteristics of flowers. They already have have invested on curating sample images. They require an automation which can create a classifier capable of determining a flower’s species from a photo. • DATA DESCRIPTION: The dataset comprises of images from 17 plant species.

• PROJECT OBJECTIVE: To experiment with various approaches to train an image classifier to predict type of flower from the image.

1. Import and Understand the data


A. Import and read oxflower17 dataset from tflearn and split into X and Y while loading.


In [38]:
!pip install tflearn
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting tflearn
  Downloading tflearn-0.5.0.tar.gz (107 kB)
     |████████████████████████████████| 107 kB 9.0 MB/s 
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from tflearn) (1.21.6)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from tflearn) (1.15.0)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from tflearn) (7.1.2)
Building wheels for collected packages: tflearn
  Building wheel for tflearn (setup.py) ... done
  Created wheel for tflearn: filename=tflearn-0.5.0-py3-none-any.whl size=127299 sha256=86dfc472b62765de782855fd2a8610a1f12341b6ebb654f0a6d91eecf921bef3
  Stored in directory: /root/.cache/pip/wheels/5f/14/2e/1d8e28cc47a5a931a2fb82438c9e37ef9246cc6a3774520271
Successfully built tflearn
Installing collected packages: tflearn
Successfully installed tflearn-0.5.0
In [39]:
from tflearn.datasets import oxflower17
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
In [40]:
X,y = oxflower17.load_data();

B. Print Number of images and shape of the images

In [41]:
print("Total Number of images " , len(X))
Total Number of images  1360

Shape of Images

In [42]:
records, height, width, channel =  X.shape
print("Shape" , X.shape)
print("Total number of records ", records)
print(f"pixel {height} x {width} x {channel}")
Shape (1360, 224, 224, 3)
Total number of records  1360
pixel 224 x 224 x 3

C. Print count of each class from y.

In [43]:
unique = np.unique(y)
print("unique values are ", unique)
print("Total number of class are ", len(unique))
unique values are  [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16]
Total number of class are  17
In [44]:
print("Each class images are ")
classRecord = pd.DataFrame(y).value_counts()
classRecord.columns =['Class', 'No of Records']
classRecord
Each class images are 
Out[44]:
0     80
9     80
15    80
14    80
13    80
12    80
11    80
10    80
8     80
1     80
7     80
6     80
5     80
4     80
3     80
2     80
16    80
dtype: int64
In [45]:
plt.figure(figsize=(10,10))
classRecord.plot(kind='barh')
plt.xlabel('Count')
plt.ylabel('Label')
Out[45]:
Text(0, 0.5, 'Label')

A. Display 5 random images


In [46]:
# generate index of each class 
def generate_random_index(totalNo, total_no_of_records, label):
  random_index = [];
  label = pd.DataFrame(label)
  label.columns = ['label']
  uniqueLabel = np.unique(label)
  for index , category in  enumerate(uniqueLabel):
    if index >=totalNo:
      break; 
    random_index.append(random.choice(label[label['label'] == category].index))
  while(len(random_index) != totalNo):
    random_index.append(random.randint(0, total_no_of_records));

  return random_index;
In [47]:
TOTAL_NUMBER_OF_RANDOM_IMAGE = 5
random_flower_index= generate_random_index(TOTAL_NUMBER_OF_RANDOM_IMAGE, len(X), y)
random_flower_X = X[random_flower_index]
random_flower_y = y[random_flower_index]
print('rows selected ')
displayImage(random_flower_X, random_flower_y)
rows selected 

B. Select any image from the dataset and assign it to a variable.

In [48]:
TOTAL_NUMBER_OF_RANDOM_IMAGE =1
random_flower_index= generate_random_index(TOTAL_NUMBER_OF_RANDOM_IMAGE, len(X), y)
random_flower_X = X[random_flower_index]
random_flower_y = y[random_flower_index]
display(random_flower_X[0].shape)
displayImage(random_flower_X, random_flower_y)
(224, 224, 3)

C. Transform the image into grayscale format and display the same

In [49]:
rgb_weights = [0.2989, 0.5870, 0.1140]
grayscale_image = np.dot(random_flower_X[...,:3], rgb_weights)
#plt.imshow(tf.squeeze(grayscale_image))
display(grayscale_image.shape)
plt.imshow(np.squeeze(grayscale_image),cmap='gray')
(1, 224, 224)
Out[49]:
<matplotlib.image.AxesImage at 0x7fb3040275d0>
In [49]:

D. Apply a filter to sharpen the image and display the image before and after sharpening.


In [50]:
img_pil = tf.keras.utils.array_to_img(random_flower_X[0])
# Apply sharp filter
sharpened1 = img_pil.filter(ImageFilter.SHARPEN);


fig, (ax1,ax2) = plt.subplots(1,2 ,figsize= (30,10))
ax1.imshow(img_pil)
ax1.set_xlabel("Original Flower Image")
ax2.imshow(sharpened1)
ax2.set_xlabel("Sharpen Flower Image");

E. Apply a filter to blur the image and display the image before and after blur. [2 Marks]


In [51]:
# Apply sharp filter
blur1 = img_pil.filter(ImageFilter.BLUR);

fig, (ax1,ax2) = plt.subplots(1,2 ,figsize= (30,10))
ax1.imshow(img_pil)
ax1.set_xlabel("Original Flower Image")
ax2.imshow(blur1)
ax2.set_xlabel("Sharpening Flower Image")
Out[51]:
Text(0.5, 0, 'Sharpening Flower Image')

F. Display all the 4 images from above questions besides each other to observe the difference. [1 Marks]


In [52]:
fig, (ax1,ax2,ax3,ax4) = plt.subplots(1,4 ,figsize= (30,10))
ax1.imshow(img_pil)
ax1.set_xlabel("Original Flower Image")
ax2.imshow(np.squeeze(grayscale_image),cmap='gray')
ax2.set_xlabel("Gray Flower Image")
ax3.imshow(sharpened1)
ax3.set_xlabel("Sharpening Flower Image")
ax4.imshow(blur1)
ax4.set_xlabel("Blur Flower Image");
# Printing each category of images.
In [53]:
plt.figure(figsize = (20,20))
col =1
for i in np.unique(y):
    plt.subplot(5,4,col)
    plt.imshow(X[i])
    plt.title(f"Flower Label: {y[i]}",{'fontsize':15})
    plt.tick_params(top=False,bottom=False,right=False,labelleft = False,labelbottom=False)
    col = col+1
plt.show()

3. Model training and Tuning:

A. Split the data into train and test with 80:20 proportion


In [54]:
X_train, X_test, y_train ,y_test = train_test_split(X,y , test_size=0.20, random_state=seed)

B. Train a model using any Supervised Learning algorithm and share performance metrics on test data


In [55]:
result = {}
In [56]:
# Flatten the layer
def flatten(X):
  Xr, Xg, Xb = [],[],[]
  for samples in X:
    r, g, b = cv2.split(samples)
    Xr.append(r.flatten())
    Xg.append(g.flatten())
    Xb.append(b.flatten())
  Xr = np.array(Xr)
  Xg = np.array(Xg)
  Xb = np.array(Xb)
  return (Xr, Xg, Xb)
In [57]:
X_train_r, X_train_g, X_train_b = flatten(X_train)
X_test_r, X_test_g, X_test_b = flatten(X_test)
In [58]:
X_train_r.shape, X_train_g.shape, X_train_b.shape,
Out[58]:
((1088, 50176), (1088, 50176), (1088, 50176))
In [59]:
X_test_r.shape, X_test_g.shape, X_test_b.shape,
Out[59]:
((272, 50176), (272, 50176), (272, 50176))
In [60]:
n_components = 200
X_train_r_pca = PCA(n_components=n_components, whiten=True).fit(X_train_r)
X_train_g_pca = PCA(n_components=n_components, whiten=True).fit(X_train_g)
X_train_b_pca = PCA(n_components=n_components, whiten=True).fit(X_train_b)

X_test_r_pca = PCA(n_components=n_components, whiten=True).fit(X_test_r)
X_test_g_pca = PCA(n_components=n_components, whiten=True).fit(X_test_g)
X_test_b_pca = PCA(n_components=n_components, whiten=True).fit(X_test_b)
In [61]:
X_train_r_pca.explained_variance_ratio_.sum(), X_train_g_pca.explained_variance_ratio_.sum(), X_train_b_pca.explained_variance_ratio_.sum()
Out[61]:
(0.84574187, 0.8352584, 0.864336)
In [62]:
X_test_r_pca.explained_variance_ratio_.sum(), X_test_g_pca.explained_variance_ratio_.sum(), X_test_b_pca.explained_variance_ratio_.sum()
Out[62]:
(0.9689964, 0.9675413, 0.9781177)
In [63]:
Xr_train_pca = X_train_r_pca.transform(X_train_r)
Xg_train_pca = X_train_g_pca.transform(X_train_g)
Xb_train_pca = X_train_b_pca.transform(X_train_b)

Xr_test_pca = X_test_r_pca.transform(X_test_r)
Xg_test_pca = X_test_g_pca.transform(X_test_g)
Xb_test_pca = X_test_b_pca.transform(X_test_b)
In [64]:
X_train_pca = np.concatenate([Xr_train_pca,Xg_train_pca,Xb_train_pca], axis=1)
X_test_pca = np.concatenate([Xr_test_pca,Xg_test_pca,Xb_test_pca], axis=1)
In [65]:
X_train_pca.shape, y_train.shape,
X_test_pca.shape, y_test.shape,
Out[65]:
((272, 600), (272,))
In [66]:
param_grid=[
            {'n_estimators':[100,200],
             'min_samples_leaf':[2,3]}
           ]
rf = RandomForestClassifier()
clf = GridSearchCV(rf, param_grid, verbose=10, n_jobs=-1)
clf.fit(X_train_pca,y_train)
Fitting 5 folds for each of 4 candidates, totalling 20 fits
Out[66]:
GridSearchCV(estimator=RandomForestClassifier(), n_jobs=-1,
             param_grid=[{'min_samples_leaf': [2, 3],
                          'n_estimators': [100, 200]}],
             verbose=10)
In [67]:
rf_score = clf.score(X_test_pca, y_test)
y_pred = clf.predict(X_test_pca)
rf_accuracy = metrics.accuracy_score(y_test, y_pred)
print(f"The predicted label is:'{rf_accuracy}'")
result['Random Forest']  = ['NA', rf_accuracy, clf]
The predicted label is:'0.17279411764705882'
In [68]:
del X_train_pca,X_test_pca

C. Train a model using Neural Network and share performance metrics on test data


In [69]:
print('Shape of X_train ' ,X_train.shape)
print('Shape of y_train ' ,y_train.shape)

print('Shape of X_test ' ,X_test.shape)
print('Shape of y_test ' ,y_test.shape)
Shape of X_train  (1088, 224, 224, 3)
Shape of y_train  (1088,)
Shape of X_test  (272, 224, 224, 3)
Shape of y_test  (272,)

Convert the label into desired CNN format

In [70]:
num_of_classes=len(np.unique(y_train))
print(num_of_classes)
y_train_cal = to_categorical(y_train,num_classes=num_of_classes)
y_test_cal = to_categorical(y_test, num_classes=num_of_classes)
17
In [71]:
train_datagen =   ImageDataGenerator(rotation_range=180,  
        zoom_range=0.3,
        width_shift_range=0.3,  
        height_shift_range=0.3, 
        shear_range=0.3, 
        horizontal_flip=True,  
        vertical_flip=True,
        fill_mode='nearest',
        )
test_datagen = ImageDataGenerator()
train_datagen.fit(X_train)
#test_datagen.fit(X_test)
batch_size = 64
train_datagen = train_datagen.flow(X_train, y_train_cal, batch_size=batch_size)

validation_set = test_datagen.flow(X_test, y_test_cal, batch_size=batch_size)
In [72]:
from keras.backend import reshape
from tensorflow import keras
tf.keras.backend.clear_session()
def build_model(hp):
  model = Sequential()
  model.add(keras.layers.InputLayer(input_shape=(224, 224, 3)))
  model.add(Flatten())
  for layer in range(hp.Int('num_layer', 1,4)):
    model.add(BatchNormalization())
    model.add(Dense(units=hp.Int("Units_"+str(layer+1), min_value=32, max_value=1024, step=32), activation= hp.Choice('activation_' +str(layer+1), ["relu"]),  kernel_initializer='he_uniform' ))
    model.add(Dropout(rate=hp.Float('dropout_' + str(layer + 1) ,
                                    min_value=0.0,
                                    max_value=0.5,
                                    step=0.1)))
  model.add(BatchNormalization())
  model.add(Dense(units=num_of_classes, activation='softmax'))
  learning_rate = hp.Float("lr", min_value=1e-4, max_value=1e-1, sampling="log")
  model.compile(optimizer=Adam(learning_rate=learning_rate),loss=CategoricalCrossentropy(),metrics=['accuracy'])
  return model
In [73]:
tuner  = Hyperband(build_model, objective=kerastuner.Objective("val_acc", direction="max"), max_epochs=15, factor=2, hyperparameters=None, directory = 'Flower_NN_Q2',project_name = 'Tuning_MODEL_NN',overwrite=True)
tuner.search_space_summary()
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/keras/layers/normalization/batch_normalization.py:532: _colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Search space summary
Default search space size: 5
num_layer (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 4, 'step': 1, 'sampling': None}
Units_1 (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 1024, 'step': 32, 'sampling': None}
activation_1 (Choice)
{'default': 'relu', 'conditions': [], 'values': ['relu'], 'ordered': False}
dropout_1 (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None}
lr (Float)
{'default': 0.0001, 'conditions': [], 'min_value': 0.0001, 'max_value': 0.1, 'step': None, 'sampling': 'log'}
In [74]:
stop_early = EarlyStopping(monitor='val_loss', mode='min', patience=7)
reduce_lr = ReduceLROnPlateau(monitor='val_loss',factor=0.1,patience=5,min_lr=0.00001,model='auto', verbose=1)
In [75]:
tuner.search(train_datagen,validation_data=(X_test,y_test_cal), steps_per_epoch=X_train.shape[0]/ batch_size)
Trial 46 Complete [00h 02m 53s]
val_acc: 0.3639705777168274

Best val_acc So Far: 0.3897058963775635
Total elapsed time: 00h 47m 49s
In [76]:
best_ANN_model = tuner.get_best_models()[0]
best_ANN_model.build(X_train.shape)
# Saving model into the memory
best_ANN_model.save('Hypertuned_ANN_Flower_Prob2.h5')
best_ANN_model.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 150528)            0         
                                                                 
 batch_normalization (BatchN  (None, 150528)           602112    
 ormalization)                                                   
                                                                 
 dense (Dense)               (None, 576)               86704704  
                                                                 
 dropout (Dropout)           (None, 576)               0         
                                                                 
 batch_normalization_1 (Batc  (None, 576)              2304      
 hNormalization)                                                 
                                                                 
 dense_1 (Dense)             (None, 17)                9809      
                                                                 
=================================================================
Total params: 87,318,929
Trainable params: 87,016,721
Non-trainable params: 302,208
_________________________________________________________________

It took 60 min to hypertuned the model. Model has been saved so that it can be reuse again with again doing hypertuning

In [77]:
# Loading model from the memory
best_ANN_model = load_model('Hypertuned_ANN_Flower_Prob2.h5')
best_ANN_model.summary()
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/init_ops.py:93: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/init_ops.py:93: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/init_ops.py:93: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 150528)            0         
                                                                 
 batch_normalization (BatchN  (None, 150528)           602112    
 ormalization)                                                   
                                                                 
 dense (Dense)               (None, 576)               86704704  
                                                                 
 dropout (Dropout)           (None, 576)               0         
                                                                 
 batch_normalization_1 (Batc  (None, 576)              2304      
 hNormalization)                                                 
                                                                 
 dense_1 (Dense)             (None, 17)                9809      
                                                                 
=================================================================
Total params: 87,318,929
Trainable params: 87,016,721
Non-trainable params: 302,208
_________________________________________________________________
In [78]:
# Train the model with 50 epochs
checkpoint_nn = ModelCheckpoint("tflearn_ANN_Flower.h5",monitor='val_acc', mode='max',verbose=1, save_best_only=True)
history_ANN_hypertuned = best_ANN_model.fit(train_datagen, validation_data=(X_test,y_test_cal), callbacks=[reduce_lr,stop_early, checkpoint_nn], epochs=50, verbose=2, steps_per_epoch=X_train.shape[0]/ 64)
Epoch 1/50

Epoch 1: val_acc improved from -inf to 0.33824, saving model to tflearn_ANN_Flower.h5
17/17 - 16s - loss: 2.4783 - acc: 0.2316 - val_loss: 2.0496 - val_acc: 0.3382 - lr: 1.7102e-04 - 16s/epoch - 914ms/step
Epoch 2/50

Epoch 2: val_acc did not improve from 0.33824
17/17 - 8s - loss: 2.4284 - acc: 0.2454 - val_loss: 2.0707 - val_acc: 0.3162 - lr: 1.7102e-04 - 8s/epoch - 450ms/step
Epoch 3/50

Epoch 3: val_acc improved from 0.33824 to 0.34191, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.4084 - acc: 0.2482 - val_loss: 1.9774 - val_acc: 0.3419 - lr: 1.7102e-04 - 14s/epoch - 819ms/step
Epoch 4/50

Epoch 4: val_acc did not improve from 0.34191
17/17 - 7s - loss: 2.3560 - acc: 0.2518 - val_loss: 2.0025 - val_acc: 0.3309 - lr: 1.7102e-04 - 7s/epoch - 438ms/step
Epoch 5/50

Epoch 5: val_acc improved from 0.34191 to 0.38971, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.2753 - acc: 0.2748 - val_loss: 1.9057 - val_acc: 0.3897 - lr: 1.7102e-04 - 14s/epoch - 806ms/step
Epoch 6/50

Epoch 6: val_acc did not improve from 0.38971
17/17 - 7s - loss: 2.2773 - acc: 0.2684 - val_loss: 1.9450 - val_acc: 0.3750 - lr: 1.7102e-04 - 7s/epoch - 423ms/step
Epoch 7/50

Epoch 7: val_acc did not improve from 0.38971
17/17 - 10s - loss: 2.2773 - acc: 0.2675 - val_loss: 1.9551 - val_acc: 0.3860 - lr: 1.7102e-04 - 10s/epoch - 591ms/step
Epoch 8/50

Epoch 8: val_acc did not improve from 0.38971
17/17 - 10s - loss: 2.2529 - acc: 0.2868 - val_loss: 1.9353 - val_acc: 0.3787 - lr: 1.7102e-04 - 10s/epoch - 588ms/step
Epoch 9/50

Epoch 9: val_acc did not improve from 0.38971
17/17 - 10s - loss: 2.2455 - acc: 0.2564 - val_loss: 1.9327 - val_acc: 0.3750 - lr: 1.7102e-04 - 10s/epoch - 590ms/step
Epoch 10/50

Epoch 10: ReduceLROnPlateau reducing learning rate to 1.7102292622439565e-05.

Epoch 10: val_acc improved from 0.38971 to 0.39338, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.2184 - acc: 0.2776 - val_loss: 1.9278 - val_acc: 0.3934 - lr: 1.7102e-04 - 14s/epoch - 818ms/step
Epoch 11/50

Epoch 11: val_acc did not improve from 0.39338
17/17 - 7s - loss: 2.1985 - acc: 0.2868 - val_loss: 1.9093 - val_acc: 0.3897 - lr: 1.7102e-05 - 7s/epoch - 413ms/step
Epoch 12/50

Epoch 12: val_acc did not improve from 0.39338
17/17 - 10s - loss: 2.1825 - acc: 0.2987 - val_loss: 1.8875 - val_acc: 0.3934 - lr: 1.7102e-05 - 10s/epoch - 591ms/step
Epoch 13/50

Epoch 13: val_acc improved from 0.39338 to 0.40074, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.1705 - acc: 0.3033 - val_loss: 1.8735 - val_acc: 0.4007 - lr: 1.7102e-05 - 14s/epoch - 813ms/step
Epoch 14/50

Epoch 14: val_acc improved from 0.40074 to 0.40809, saving model to tflearn_ANN_Flower.h5
17/17 - 12s - loss: 2.1566 - acc: 0.3061 - val_loss: 1.8587 - val_acc: 0.4081 - lr: 1.7102e-05 - 12s/epoch - 717ms/step
Epoch 15/50

Epoch 15: val_acc improved from 0.40809 to 0.41544, saving model to tflearn_ANN_Flower.h5
17/17 - 12s - loss: 2.0997 - acc: 0.2969 - val_loss: 1.8482 - val_acc: 0.4154 - lr: 1.7102e-05 - 12s/epoch - 721ms/step
Epoch 16/50

Epoch 16: val_acc improved from 0.41544 to 0.41912, saving model to tflearn_ANN_Flower.h5
17/17 - 12s - loss: 2.2073 - acc: 0.2849 - val_loss: 1.8389 - val_acc: 0.4191 - lr: 1.7102e-05 - 12s/epoch - 682ms/step
Epoch 17/50

Epoch 17: val_acc did not improve from 0.41912
17/17 - 7s - loss: 2.1486 - acc: 0.2987 - val_loss: 1.8390 - val_acc: 0.4118 - lr: 1.7102e-05 - 7s/epoch - 423ms/step
Epoch 18/50

Epoch 18: val_acc did not improve from 0.41912
17/17 - 10s - loss: 2.1048 - acc: 0.3088 - val_loss: 1.8370 - val_acc: 0.4191 - lr: 1.7102e-05 - 10s/epoch - 587ms/step
Epoch 19/50

Epoch 19: val_acc did not improve from 0.41912
17/17 - 10s - loss: 2.1027 - acc: 0.3217 - val_loss: 1.8268 - val_acc: 0.4118 - lr: 1.7102e-05 - 10s/epoch - 587ms/step
Epoch 20/50

Epoch 20: val_acc did not improve from 0.41912
17/17 - 10s - loss: 2.1423 - acc: 0.3244 - val_loss: 1.8219 - val_acc: 0.4154 - lr: 1.7102e-05 - 10s/epoch - 587ms/step
Epoch 21/50

Epoch 21: val_acc did not improve from 0.41912
17/17 - 10s - loss: 2.1392 - acc: 0.3171 - val_loss: 1.8195 - val_acc: 0.4154 - lr: 1.7102e-05 - 10s/epoch - 586ms/step
Epoch 22/50

Epoch 22: val_acc improved from 0.41912 to 0.42647, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.1342 - acc: 0.3153 - val_loss: 1.8187 - val_acc: 0.4265 - lr: 1.7102e-05 - 14s/epoch - 810ms/step
Epoch 23/50

Epoch 23: val_acc improved from 0.42647 to 0.43015, saving model to tflearn_ANN_Flower.h5
17/17 - 12s - loss: 2.1562 - acc: 0.3061 - val_loss: 1.8168 - val_acc: 0.4301 - lr: 1.7102e-05 - 12s/epoch - 690ms/step
Epoch 24/50

Epoch 24: val_acc did not improve from 0.43015
17/17 - 6s - loss: 2.1537 - acc: 0.3143 - val_loss: 1.8121 - val_acc: 0.4301 - lr: 1.7102e-05 - 6s/epoch - 376ms/step
Epoch 25/50

Epoch 25: val_acc did not improve from 0.43015
17/17 - 10s - loss: 2.1235 - acc: 0.3088 - val_loss: 1.8035 - val_acc: 0.4265 - lr: 1.7102e-05 - 10s/epoch - 592ms/step
Epoch 26/50

Epoch 26: val_acc improved from 0.43015 to 0.43382, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.1588 - acc: 0.3033 - val_loss: 1.7948 - val_acc: 0.4338 - lr: 1.7102e-05 - 14s/epoch - 830ms/step
Epoch 27/50

Epoch 27: val_acc did not improve from 0.43382
17/17 - 7s - loss: 2.0745 - acc: 0.3125 - val_loss: 1.7929 - val_acc: 0.4228 - lr: 1.7102e-05 - 7s/epoch - 405ms/step
Epoch 28/50

Epoch 28: val_acc did not improve from 0.43382
17/17 - 10s - loss: 2.0833 - acc: 0.3107 - val_loss: 1.7941 - val_acc: 0.4228 - lr: 1.7102e-05 - 10s/epoch - 588ms/step
Epoch 29/50

Epoch 29: val_acc did not improve from 0.43382
17/17 - 10s - loss: 2.0907 - acc: 0.3143 - val_loss: 1.7921 - val_acc: 0.4265 - lr: 1.7102e-05 - 10s/epoch - 589ms/step
Epoch 30/50

Epoch 30: val_acc did not improve from 0.43382
17/17 - 11s - loss: 2.0996 - acc: 0.3116 - val_loss: 1.7912 - val_acc: 0.4265 - lr: 1.7102e-05 - 11s/epoch - 674ms/step
Epoch 31/50

Epoch 31: val_acc improved from 0.43382 to 0.43750, saving model to tflearn_ANN_Flower.h5
17/17 - 15s - loss: 2.1153 - acc: 0.3217 - val_loss: 1.7930 - val_acc: 0.4375 - lr: 1.7102e-05 - 15s/epoch - 872ms/step
Epoch 32/50

Epoch 32: val_acc did not improve from 0.43750
17/17 - 7s - loss: 2.0842 - acc: 0.3290 - val_loss: 1.7996 - val_acc: 0.4191 - lr: 1.7102e-05 - 7s/epoch - 430ms/step
Epoch 33/50

Epoch 33: val_acc did not improve from 0.43750
17/17 - 10s - loss: 2.1045 - acc: 0.3061 - val_loss: 1.8025 - val_acc: 0.4301 - lr: 1.7102e-05 - 10s/epoch - 590ms/step
Epoch 34/50

Epoch 34: val_acc did not improve from 0.43750
17/17 - 10s - loss: 2.1016 - acc: 0.3244 - val_loss: 1.8035 - val_acc: 0.4265 - lr: 1.7102e-05 - 10s/epoch - 588ms/step
Epoch 35/50

Epoch 35: ReduceLROnPlateau reducing learning rate to 1e-05.

Epoch 35: val_acc did not improve from 0.43750
17/17 - 10s - loss: 2.0637 - acc: 0.3051 - val_loss: 1.7976 - val_acc: 0.4228 - lr: 1.7102e-05 - 10s/epoch - 587ms/step
Epoch 36/50

Epoch 36: val_acc did not improve from 0.43750
17/17 - 10s - loss: 2.0348 - acc: 0.3281 - val_loss: 1.7970 - val_acc: 0.4265 - lr: 1.0000e-05 - 10s/epoch - 587ms/step
Epoch 37/50

Epoch 37: val_acc improved from 0.43750 to 0.44118, saving model to tflearn_ANN_Flower.h5
17/17 - 14s - loss: 2.0553 - acc: 0.3143 - val_loss: 1.7969 - val_acc: 0.4412 - lr: 1.0000e-05 - 14s/epoch - 822ms/step
In [79]:
y_pre = best_ANN_model.predict(X_test)
y_pred_final=[]
for i in y_pre:
  y_pred_final.append(np.argmax(i))
/usr/local/lib/python3.7/dist-packages/keras/engine/training_v1.py:2079: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
  updates=self.state_updates,
In [80]:
loss, accuracy = best_ANN_model.evaluate(X_test, y_test_cal)
result['ANN']  = [loss, accuracy, best_ANN_model]
print('loss in Testing data ', loss)
print('Accuracy in Testing data ', accuracy)
loss in Testing data  1.7479017762576832
Accuracy in Testing data  0.44117647

Accuracy is 44% . Lets try CNN

In [81]:
# Printing the confusion metrics
cm = confusion_matrix(y_pred_final,y_test)
plt.figure(figsize=(10,7))
sns.heatmap(cm,annot=True,fmt='d')
plt.xlabel('Truth')
plt.ylabel('Predicted')
Out[81]:
Text(69.0, 0.5, 'Predicted')

Observation: Model is bad in predicting 14,12,8, 6,3,1,0.

Printing the classification report

In [82]:
print(classification_report(y_test, y_pred_final))
              precision    recall  f1-score   support

           0       0.21      0.20      0.21        15
           1       0.44      0.22      0.30        18
           2       0.32      0.50      0.39        12
           3       0.38      0.40      0.39        20
           4       0.32      0.40      0.36        20
           5       0.40      0.35      0.38        17
           6       0.21      0.27      0.24        15
           7       0.35      0.67      0.46        12
           8       0.22      0.20      0.21        10
           9       0.45      0.50      0.48        20
          10       0.62      0.57      0.59        14
          11       0.43      0.43      0.43        14
          12       0.62      0.57      0.59        14
          13       0.64      0.53      0.58        17
          14       0.45      0.29      0.36        17
          15       0.67      0.43      0.52        14
          16       0.86      0.83      0.84        23

    accuracy                           0.44       272
   macro avg       0.45      0.43      0.43       272
weighted avg       0.46      0.44      0.44       272

# F1 score is less for each class. Overall accuracy  is 44%
In [83]:
#  displaying the Accuracy and Loss of model
def show_final_history(history):
    fig, ax = plt.subplots(1,2,figsize=(15,5))
    ax[0].set_title("Loss")
    ax[1].set_title("Accuracy")
    ax[0].plot(history.history["loss"],label="Loss")
    ax[0].plot(history.history["val_loss"],label="Test Loss")
    ax[1].plot(history.history["acc"],label="Accuracy")
    ax[1].plot(history.history["val_acc"],label="Test Accuracy")
    
    ax[0].legend(loc="upper right")
    ax[1].legend(loc="lower right")
In [84]:
show_final_history(history_ANN_hypertuned)

D. Train a model using a basic CNN and share performance metrics on test data


In [85]:
tf.keras.backend.clear_session()
# Hypertuning CNN model
In [86]:
def build_model(hp):
    model=Sequential();
    model.add(tf.keras.Input(shape=(224,224,3)))
    for layer in range(hp.Int('num_conv_layer', 1,5)):
      model.add(BatchNormalization())
      model.add(Conv2D(filters = hp.Int('conv '+str(layer+1)+'_filter', min_value=32, max_value=128, step=16),kernel_size=hp.Choice('Conv_Kernel_'+str(layer+1), [3,5,7]), padding=hp.Choice('Conv_padding_'+str(layer+1), ['valid','same']) ,
                       activation= "relu" ))
      model.add(BatchNormalization())
      model.add(MaxPooling2D())
  
    model.add(GlobalAveragePooling2D())
    for layer in range(hp.Int('num__dense_layer', 1,4)):
      model.add(BatchNormalization())
      model.add(Dropout(rate=hp.Float('dropout_' + str(layer + 1) ,
                                      min_value=0.0,
                                      max_value=0.5,
                                      step=0.1)))
      model.add(Dense(units=hp.Int("Units_"+str(layer+1), min_value=32, max_value=512, step=64), activation= "relu",  kernel_initializer='he_uniform'))
      model.add(BatchNormalization())
      
    model.add(Dense(units=num_of_classes,activation='softmax'))
    learning_rate = hp.Float("lr", min_value=1e-4, max_value=1e-1, sampling="log")
    model.compile(optimizer=Adam(learning_rate=learning_rate),
              loss=CategoricalCrossentropy(),
              metrics=['accuracy'])
    return model
In [87]:
tuner  = Hyperband(build_model, objective=kerastuner.Objective("val_acc", direction="max"), max_epochs=15, factor=2, hyperparameters=None, directory = 'Flower_NN_Q2',project_name = 'TUNING_MODEL_CNN',overwrite=True)
tuner.search_space_summary()
Search space summary
Default search space size: 8
num_conv_layer (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 5, 'step': 1, 'sampling': None}
conv 1_filter (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 128, 'step': 16, 'sampling': None}
Conv_Kernel_1 (Choice)
{'default': 3, 'conditions': [], 'values': [3, 5, 7], 'ordered': True}
Conv_padding_1 (Choice)
{'default': 'valid', 'conditions': [], 'values': ['valid', 'same'], 'ordered': False}
num__dense_layer (Int)
{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 4, 'step': 1, 'sampling': None}
dropout_1 (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.5, 'step': 0.1, 'sampling': None}
Units_1 (Int)
{'default': None, 'conditions': [], 'min_value': 32, 'max_value': 512, 'step': 64, 'sampling': None}
lr (Float)
{'default': 0.0001, 'conditions': [], 'min_value': 0.0001, 'max_value': 0.1, 'step': None, 'sampling': 'log'}
In [88]:
print("images shape: ", X_train.shape)
print("classes shape: ", y_train.shape)
images shape:  (1088, 224, 224, 3)
classes shape:  (1088,)
In [89]:
# tuner.search(train_datagen,
#                     steps_per_epoch=(X_train.shape[0]/batch_size),
#                     validation_data=(X_test, y_test_cal) 
#              )
In [90]:
# best_CNN_model = tuner.get_best_models()[0]
# best_CNN_model.build(X_train.shape)
# best_CNN_model.summary()
# best_CNN_model.save('Hypertuned_Flower_CNN_Prob2.h5')
# best_CNN_model.summary()

It is taking more than 1 hours. So i am using the already Saved model

In [91]:
best_CNN_model = load_model('Hypertuned_Flower_CNN_Prob2.h5')
best_CNN_model.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 batch_normalization (BatchN  (None, 224, 224, 3)      12        
 ormalization)                                                   
                                                                 
 conv2d (Conv2D)             (None, 224, 224, 80)      6080      
                                                                 
 batch_normalization_1 (Batc  (None, 224, 224, 80)     320       
 hNormalization)                                                 
                                                                 
 max_pooling2d (MaxPooling2D  (None, 112, 112, 80)     0         
 )                                                               
                                                                 
 batch_normalization_2 (Batc  (None, 112, 112, 80)     320       
 hNormalization)                                                 
                                                                 
 conv2d_1 (Conv2D)           (None, 112, 112, 48)      188208    
                                                                 
 batch_normalization_3 (Batc  (None, 112, 112, 48)     192       
 hNormalization)                                                 
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 56, 56, 48)       0         
 2D)                                                             
                                                                 
 batch_normalization_4 (Batc  (None, 56, 56, 48)       192       
 hNormalization)                                                 
                                                                 
 conv2d_2 (Conv2D)           (None, 50, 50, 32)        75296     
                                                                 
 batch_normalization_5 (Batc  (None, 50, 50, 32)       128       
 hNormalization)                                                 
                                                                 
 max_pooling2d_2 (MaxPooling  (None, 25, 25, 32)       0         
 2D)                                                             
                                                                 
 batch_normalization_6 (Batc  (None, 25, 25, 32)       128       
 hNormalization)                                                 
                                                                 
 conv2d_3 (Conv2D)           (None, 23, 23, 32)        9248      
                                                                 
 batch_normalization_7 (Batc  (None, 23, 23, 32)       128       
 hNormalization)                                                 
                                                                 
 max_pooling2d_3 (MaxPooling  (None, 11, 11, 32)       0         
 2D)                                                             
                                                                 
 global_average_pooling2d (G  (None, 32)               0         
 lobalAveragePooling2D)                                          
                                                                 
 batch_normalization_8 (Batc  (None, 32)               128       
 hNormalization)                                                 
                                                                 
 dropout (Dropout)           (None, 32)                0         
                                                                 
 dense (Dense)               (None, 352)               11616     
                                                                 
 batch_normalization_9 (Batc  (None, 352)              1408      
 hNormalization)                                                 
                                                                 
 batch_normalization_10 (Bat  (None, 352)              1408      
 chNormalization)                                                
                                                                 
 dropout_1 (Dropout)         (None, 352)               0         
                                                                 
 dense_1 (Dense)             (None, 416)               146848    
                                                                 
 batch_normalization_11 (Bat  (None, 416)              1664      
 chNormalization)                                                
                                                                 
 dense_2 (Dense)             (None, 17)                7089      
                                                                 
=================================================================
Total params: 450,413
Trainable params: 447,399
Non-trainable params: 3,014
_________________________________________________________________
In [92]:
#Training the model with 50 epochs
checkpoint_cnn = ModelCheckpoint("tflearn_CNN_.h5",monitor='val_acc',
                             mode='max',verbose=1, save_best_only=True)
history_hypertuned = best_CNN_model.fit(train_datagen, validation_data=(X_test, y_test_cal), callbacks=[reduce_lr, stop_early,checkpoint_cnn], epochs=50,  verbose=2)
#history_hypertuned = best_CNN_model.fit(X_train, y_train_cal, validation_data=(X_test, y_test_cal), callbacks=[reduce_lr, stop_early,checkpoint_cnn], epochs=100,  verbose=2, batch_size=64)
Epoch 1/50
/usr/local/lib/python3.7/dist-packages/keras/engine/training_v1.py:2057: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
  updates = self.state_updates
Epoch 1: val_acc improved from -inf to 0.26103, saving model to tflearn_CNN_.h5
17/17 - 13s - loss: 1.7160 - acc: 0.4430 - val_loss: 4.2869 - val_acc: 0.2610 - lr: 0.0046 - 13s/epoch - 742ms/step
Epoch 2/50

Epoch 2: val_acc improved from 0.26103 to 0.33456, saving model to tflearn_CNN_.h5
17/17 - 9s - loss: 1.5600 - acc: 0.4632 - val_loss: 4.4501 - val_acc: 0.3346 - lr: 0.0046 - 9s/epoch - 550ms/step
Epoch 3/50

Epoch 3: val_acc improved from 0.33456 to 0.35662, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.4599 - acc: 0.5221 - val_loss: 4.1043 - val_acc: 0.3566 - lr: 0.0046 - 10s/epoch - 603ms/step
Epoch 4/50

Epoch 4: val_acc improved from 0.35662 to 0.37500, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.4004 - acc: 0.5221 - val_loss: 4.2635 - val_acc: 0.3750 - lr: 0.0046 - 10s/epoch - 603ms/step
Epoch 5/50

Epoch 5: val_acc did not improve from 0.37500
17/17 - 10s - loss: 1.4352 - acc: 0.5064 - val_loss: 3.1416 - val_acc: 0.3750 - lr: 0.0046 - 10s/epoch - 599ms/step
Epoch 6/50

Epoch 6: val_acc improved from 0.37500 to 0.39338, saving model to tflearn_CNN_.h5
17/17 - 12s - loss: 1.3839 - acc: 0.5211 - val_loss: 2.8289 - val_acc: 0.3934 - lr: 0.0046 - 12s/epoch - 698ms/step
Epoch 7/50

Epoch 7: val_acc did not improve from 0.39338
17/17 - 12s - loss: 1.3403 - acc: 0.5358 - val_loss: 2.5255 - val_acc: 0.3934 - lr: 0.0046 - 12s/epoch - 708ms/step
Epoch 8/50

Epoch 8: val_acc did not improve from 0.39338
17/17 - 10s - loss: 1.2478 - acc: 0.5671 - val_loss: 3.1531 - val_acc: 0.3934 - lr: 0.0046 - 10s/epoch - 613ms/step
Epoch 9/50

Epoch 9: val_acc did not improve from 0.39338
17/17 - 10s - loss: 1.2242 - acc: 0.5827 - val_loss: 2.2112 - val_acc: 0.3934 - lr: 0.0046 - 10s/epoch - 605ms/step
Epoch 10/50

Epoch 10: val_acc did not improve from 0.39338
17/17 - 10s - loss: 1.2127 - acc: 0.5643 - val_loss: 3.1133 - val_acc: 0.3713 - lr: 0.0046 - 10s/epoch - 616ms/step
Epoch 11/50

Epoch 11: val_acc improved from 0.39338 to 0.43382, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.2401 - acc: 0.5781 - val_loss: 2.3785 - val_acc: 0.4338 - lr: 0.0046 - 10s/epoch - 607ms/step
Epoch 12/50

Epoch 12: val_acc improved from 0.43382 to 0.44118, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.2017 - acc: 0.5809 - val_loss: 2.0408 - val_acc: 0.4412 - lr: 0.0046 - 10s/epoch - 609ms/step
Epoch 13/50

Epoch 13: val_acc improved from 0.44118 to 0.51103, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.1836 - acc: 0.5947 - val_loss: 1.6878 - val_acc: 0.5110 - lr: 0.0046 - 10s/epoch - 600ms/step
Epoch 14/50

Epoch 14: val_acc did not improve from 0.51103
17/17 - 10s - loss: 1.1109 - acc: 0.6149 - val_loss: 2.5000 - val_acc: 0.4265 - lr: 0.0046 - 10s/epoch - 606ms/step
Epoch 15/50

Epoch 15: val_acc improved from 0.51103 to 0.52206, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 1.1046 - acc: 0.6195 - val_loss: 1.7211 - val_acc: 0.5221 - lr: 0.0046 - 10s/epoch - 612ms/step
Epoch 16/50

Epoch 16: val_acc improved from 0.52206 to 0.57721, saving model to tflearn_CNN_.h5
17/17 - 11s - loss: 1.1000 - acc: 0.6305 - val_loss: 1.6887 - val_acc: 0.5772 - lr: 0.0046 - 11s/epoch - 650ms/step
Epoch 17/50

Epoch 17: val_acc did not improve from 0.57721
17/17 - 10s - loss: 1.0968 - acc: 0.6195 - val_loss: 1.6782 - val_acc: 0.5257 - lr: 0.0046 - 10s/epoch - 612ms/step
Epoch 18/50

Epoch 18: val_acc did not improve from 0.57721
17/17 - 10s - loss: 1.0273 - acc: 0.6314 - val_loss: 2.2235 - val_acc: 0.5037 - lr: 0.0046 - 10s/epoch - 610ms/step
Epoch 19/50

Epoch 19: val_acc did not improve from 0.57721
17/17 - 12s - loss: 1.0955 - acc: 0.6158 - val_loss: 1.5517 - val_acc: 0.5404 - lr: 0.0046 - 12s/epoch - 678ms/step
Epoch 20/50

Epoch 20: val_acc did not improve from 0.57721
17/17 - 11s - loss: 1.0547 - acc: 0.6471 - val_loss: 1.8527 - val_acc: 0.5037 - lr: 0.0046 - 11s/epoch - 668ms/step
Epoch 21/50

Epoch 21: val_acc improved from 0.57721 to 0.59926, saving model to tflearn_CNN_.h5
17/17 - 11s - loss: 0.9908 - acc: 0.6443 - val_loss: 1.3857 - val_acc: 0.5993 - lr: 0.0046 - 11s/epoch - 625ms/step
Epoch 22/50

Epoch 22: val_acc did not improve from 0.59926
17/17 - 10s - loss: 1.0025 - acc: 0.6645 - val_loss: 1.1814 - val_acc: 0.5809 - lr: 0.0046 - 10s/epoch - 598ms/step
Epoch 23/50

Epoch 23: val_acc did not improve from 0.59926
17/17 - 10s - loss: 0.9710 - acc: 0.6792 - val_loss: 1.4068 - val_acc: 0.5735 - lr: 0.0046 - 10s/epoch - 596ms/step
Epoch 24/50

Epoch 24: val_acc did not improve from 0.59926
17/17 - 10s - loss: 0.9667 - acc: 0.6710 - val_loss: 1.3433 - val_acc: 0.5882 - lr: 0.0046 - 10s/epoch - 602ms/step
Epoch 25/50

Epoch 25: val_acc did not improve from 0.59926
17/17 - 10s - loss: 1.0027 - acc: 0.6553 - val_loss: 1.5037 - val_acc: 0.5294 - lr: 0.0046 - 10s/epoch - 606ms/step
Epoch 26/50

Epoch 26: val_acc improved from 0.59926 to 0.63235, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 0.9485 - acc: 0.6765 - val_loss: 1.2314 - val_acc: 0.6324 - lr: 0.0046 - 10s/epoch - 607ms/step
Epoch 27/50

Epoch 27: ReduceLROnPlateau reducing learning rate to 0.00045668636448681355.

Epoch 27: val_acc did not improve from 0.63235
17/17 - 10s - loss: 0.8920 - acc: 0.6958 - val_loss: 1.2102 - val_acc: 0.6287 - lr: 0.0046 - 10s/epoch - 604ms/step
Epoch 28/50

Epoch 28: val_acc improved from 0.63235 to 0.68015, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 0.8349 - acc: 0.7059 - val_loss: 0.9548 - val_acc: 0.6801 - lr: 4.5669e-04 - 10s/epoch - 609ms/step
Epoch 29/50

Epoch 29: val_acc improved from 0.68015 to 0.70221, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 0.7750 - acc: 0.7307 - val_loss: 0.9360 - val_acc: 0.7022 - lr: 4.5669e-04 - 10s/epoch - 612ms/step
Epoch 30/50

Epoch 30: val_acc improved from 0.70221 to 0.72426, saving model to tflearn_CNN_.h5
17/17 - 10s - loss: 0.6934 - acc: 0.7528 - val_loss: 0.8840 - val_acc: 0.7243 - lr: 4.5669e-04 - 10s/epoch - 610ms/step
Epoch 31/50

Epoch 31: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6981 - acc: 0.7656 - val_loss: 0.9080 - val_acc: 0.6949 - lr: 4.5669e-04 - 10s/epoch - 607ms/step
Epoch 32/50

Epoch 32: val_acc did not improve from 0.72426
17/17 - 11s - loss: 0.6672 - acc: 0.7675 - val_loss: 0.9166 - val_acc: 0.7059 - lr: 4.5669e-04 - 11s/epoch - 667ms/step
Epoch 33/50

Epoch 33: val_acc did not improve from 0.72426
17/17 - 12s - loss: 0.6444 - acc: 0.7776 - val_loss: 0.8918 - val_acc: 0.7059 - lr: 4.5669e-04 - 12s/epoch - 686ms/step
Epoch 34/50

Epoch 34: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6975 - acc: 0.7574 - val_loss: 0.9284 - val_acc: 0.6949 - lr: 4.5669e-04 - 10s/epoch - 600ms/step
Epoch 35/50

Epoch 35: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6953 - acc: 0.7537 - val_loss: 0.8813 - val_acc: 0.6985 - lr: 4.5669e-04 - 10s/epoch - 601ms/step
Epoch 36/50

Epoch 36: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6668 - acc: 0.7647 - val_loss: 0.8902 - val_acc: 0.7059 - lr: 4.5669e-04 - 10s/epoch - 604ms/step
Epoch 37/50

Epoch 37: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6501 - acc: 0.7730 - val_loss: 0.9138 - val_acc: 0.6949 - lr: 4.5669e-04 - 10s/epoch - 598ms/step
Epoch 38/50

Epoch 38: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6523 - acc: 0.7748 - val_loss: 0.8982 - val_acc: 0.7059 - lr: 4.5669e-04 - 10s/epoch - 606ms/step
Epoch 39/50

Epoch 39: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6556 - acc: 0.7647 - val_loss: 0.9144 - val_acc: 0.7059 - lr: 4.5669e-04 - 10s/epoch - 595ms/step
Epoch 40/50

Epoch 40: ReduceLROnPlateau reducing learning rate to 4.566863644868136e-05.

Epoch 40: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6409 - acc: 0.7730 - val_loss: 0.9556 - val_acc: 0.6875 - lr: 4.5669e-04 - 10s/epoch - 603ms/step
Epoch 41/50

Epoch 41: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6582 - acc: 0.7767 - val_loss: 0.9384 - val_acc: 0.6838 - lr: 4.5669e-05 - 10s/epoch - 609ms/step
Epoch 42/50

Epoch 42: val_acc did not improve from 0.72426
17/17 - 10s - loss: 0.6491 - acc: 0.7730 - val_loss: 0.9088 - val_acc: 0.6949 - lr: 4.5669e-05 - 10s/epoch - 602ms/step
In [93]:
y_pre = best_CNN_model.predict(X_test)
y_pred_final=[]
for i in y_pre:
  y_pred_final.append(np.argmax(i))
/usr/local/lib/python3.7/dist-packages/keras/engine/training_v1.py:2079: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
  updates=self.state_updates,
In [94]:
loss, accuracy = best_CNN_model.evaluate(X_test, y_test_cal)
result['CNN']  = [loss, accuracy, best_CNN_model]
print('loss in Testing data ', loss)
print('Accuracy in Testing data ', accuracy)
loss in Testing data  0.8670400591457591
Accuracy in Testing data  0.69485295
In [95]:
cm = confusion_matrix(y_pred_final,y_test)
plt.figure(figsize=(10,7))
sns.heatmap(cm,annot=True,fmt='d')
plt.xlabel('Truth')
plt.ylabel('Predicted')
Out[95]:
Text(69.0, 0.5, 'Predicted')
In [96]:
print(classification_report(y_test, y_pred_final))
              precision    recall  f1-score   support

           0       0.29      0.40      0.33        15
           1       0.67      0.67      0.67        18
           2       0.79      0.92      0.85        12
           3       0.62      0.65      0.63        20
           4       0.67      0.60      0.63        20
           5       0.62      0.88      0.73        17
           6       1.00      0.40      0.57        15
           7       0.69      0.92      0.79        12
           8       0.78      0.70      0.74        10
           9       0.74      0.70      0.72        20
          10       0.86      0.86      0.86        14
          11       0.77      0.71      0.74        14
          12       0.90      0.64      0.75        14
          13       0.88      0.82      0.85        17
          14       0.33      0.29      0.31        17
          15       1.00      0.86      0.92        14
          16       0.77      0.87      0.82        23

    accuracy                           0.69       272
   macro avg       0.73      0.70      0.70       272
weighted avg       0.72      0.69      0.69       272

In [97]:
show_final_history(history_hypertuned)
  • Loss for training data is less than testing,
  • Accuracy of Test is less than training..
  • Need to add more data or dropout layer to avoid overfitting.

Loss for training is less than testing.

In [98]:
#Printing the result of all model and take the better one
result1 = pd.DataFrame(np.array(list(result.values()))[:,:-1],    # make a dataframe out of the metrics from result dictionary 
                       columns= ['Loss','accuracy'],
                      index= result.keys())   # use the model names as index

result1.index.name = 'Model'   # name the index of the result1 dataframe as 'Model'
result1
Out[98]:
Loss accuracy
Model
Random Forest NA 0.172794
ANN 1.747902 0.441176
CNN 0.86704 0.694853

Observation:

  • 1) CNN model is giving the best accuracy.
  • 2) Loss is very less
  • 3) Moving with CNN to predict the images

E. Predict the class/label of image ‘Prediction.jpg’ using best performing model and share predicted label.


In [99]:
model = result['CNN'][2]
In [100]:
img  = WORK_DIR + '/Prediction.jpg'
predicted_img = cv2.imread(img)
predicted_img = cv2.cvtColor(predicted_img, cv2.COLOR_BGR2RGB)
predicted_img = predicted_img/255
predicted_img = np.expand_dims(predicted_img, axis = 0)
#img =img*1/255.0
#Check the size of the Image array again
print('After expand_dims: '+ str(predicted_img.shape))
result = model.predict(predicted_img)
predicted_index = np.argmax(result[0]);
plt.imshow(np.squeeze(predicted_img))
plt.suptitle('Predicted label '+str(predicted_index))
plt.show();
After expand_dims: (1, 224, 224, 3)

Obseravation : The image are predicted with LABEL 2